Recent comments in /f/MachineLearning

cdsmith t1_j6tg9z7 wrote

Awesome question! I definitely laughed.

The serious answer that the GitHub link clarifies is that the model is semi-unsupervised. That means they have a lot of data, but only some of it is labeled. Presumably, the labeled data is all negative because we understand its natural origin. So effectively this becomes almost an anomaly detection sort of thing, looking for data that is least like the known natural signals.

Even if it just directs scientists to look at new natural phenomena, this sounds like a valuable task.

17

deathisnear t1_j6swwf9 wrote

You can implement alpha beta pruning to further reduce the number of actions evaluated or look at Monte Carlo tree search as a potential option in terms of scalability (this combined with deep learning was used for Alpha Go). Games such as Fire Emblem have a similar setup and they definitely are not using RL for such a case and they tend to have reasonable performance.

1

uchi__mata t1_j6srpnd wrote

I don't see prompt construction obviating the need for coding skills, even as the prompts improve I still think you're going to want knowledgeable humans to review the scripts before using them in critical apps, but I do think tools like GPT will rapidly speed up prototyping and eliminate boilerplate dev for most engineers.

That said, model APIs strike me as a much more likely disruptor of workaday software dev because as they prove themselves out it'll just make financial sense for firms to have fewer people creating bespoke models vs pulling stuff off the shelf and modifying it as needed. In this world data science largely becomes an orchestration task with ML ops/data engineering + understanding of business need and available data being translated into ML pipeline creation to solve problems. People working directly on model creation from scratch would mostly be academics and highly skilled CS/stats/math PhDs working at a handful of large tech companies and model API firms. This seems like the most probable future to me as almost every innovation in tech goes this route eventually.

Basically, if a task doesn't require deep understanding of business needs, it's subject to commoditization.

82

jtpaquet OP t1_j6srph6 wrote

Ok so basically, I can understand the search algorithm, it is evaluating all possible move, but in the case of the game, there would be a lot of possibilities to evaluate.Movement: if I can move 7 squares, I have 113 possibilities (in a map with no walls).movement far from enemy: 113/4 ~ 29movement but I stay at the same distance from enemy: 113/2 ~ 56movement closer to the enemy: 113/4 ~ 28Attack: I'll count average roll because damage is a bit random in this game. I would say I have 10 average rolls I could do on the enemy. If I'm closer I have more possibilities to attack.From afar: 3 ways of attacking.From medium distance: 5 ways of attackingFrom close range: 10 ways of attackingAn average game would have 5 turns for each player, so 10 turns.Game outcomes = (movement_far x 3 + move_medium x 5 + move_close x 10)^10 = 1.28 x 10^28 possibilities

Let's eleminate some dumb cases: I'll keep 4 moves where I stay at a distance, 10 moves where I stay at medium range and 10 moves where I stay at close range.Game outcomes = (4 x 3 + 10 x 5 + 10 x 10)^10 = 1.25 x 10^22 possibilities.
Trying to find the state only 4 turns forward would lead to ~700 million possiblities to evaluate.

Even with pruning, considering this is the base case, and that there is a lot more ways to move on the map because different characters have different spells, I don't know if it would be suitable. It seemed to me that RL was the best alternative because it could potentially see if there is walls near the enemy and optimize based on details like that over time.

However, I didn't give full context on the game so my bad. The map would also change and the board is maximum 32x32 but typically 16x16. Let me know if there is something I don't understand or if the high number of possibilities is not a problem.

1

wind_dude t1_j6sj0ix wrote

I solved a similar issue by building a knowledge graph. It took some manual curation and starting with a good base, but suggestions for misspelling and alternates were suggested by comparing vectors. The suggester runs as a batch with new entities after my ETL batch is done.

3